One of the main reasons our customers choose CockroachDB is the easy horizontal scalability it offers, while maintaining data consistency with serializable isolation. This combination lets customers run critical OLTP workloads, like financial ledgers and e-commerce shopping carts, at large scale without the hassle of legacy sharding. With every release, we make significant investments in improving CockroachDB’s performance and scale. We measure CockroachDB’s performance through many diverse tests, including the industry-standard TPC-C benchmark to track our progress across releases.
Our latest version, CockroachDB 20.2, passed 140K warehouses with a maximum throughput of 1.7M transactions per minute (tpmC) on TPC-C. This represents a 40% improvement with the same resources as compared to the results previously reported with CockroachDB 19.2 in this post. Additionally, CockroachDB 20.2 was able to load TPC-C 140K in less than 3 hours compared to the ~20 hours it took to load TPC-C 100K in 19.2. This improvement was the result of faster bulk-data loading, which built upon work in Pebble, CockroachDB’s new storage engine.
If this were an official TPC-C run, it would be the 20th largest run on the TPC-C leaderboard.
Not only has CockroachDB 20.2 reached a higher maximum warehouse on TPC-C than in any previous release, but CockroachDB is also more efficient with the same cluster resources for workloads of varying sizes. This allows our customers to handle larger workloads on their clusters, saving costs.
What is the TPC-C benchmark?
TPC-C is the industry-standard OLTP benchmark which simulates an e-commerce or retail company. Created in 1992, TPC-C has withstood the test of time and remains the most mature industry benchmark for OLTP workloads, and the only objective comparison for evaluating OLTP performance. In its own words, TPC-C:
“…involves a mix of five concurrent transactions of different types and complexity either executed on-line or queued for deferred execution. The database is comprised of nine types of tables with a wide range of record and population sizes. While the benchmark portrays the activity of a wholesale supplier, TPC-C is not limited to the activity of any particular business segment, but, rather represents any industry that must manage, sell, or distribute a product or service.”
As a result, TPC-C includes create, read, update, and delete (e.g., CRUD) queries, basic joins, and other SQL statements used to administer mission-critical transactional workloads. It includes detailed specifications for concurrency and workload contention.
TPC-C 140,000: CockroachDB 20.2 creates greater efficiency with the same cluster size
On commodity AWS hardware, CockroachDB 20.2 allows you to do more with the same cluster resources as is evident by the higher max warehouses metric that TPC-C measures. In 20.2, more performant foreign key checks and lookup joins contribute to greater efficiency. For foreign keys, indexes are no longer required on the referencing column. Therefore, writes during the TPC-C workload won’t incur the performance overhead of updating indexes every time.
Reproduce these results
We’ve updated our performance page with the steps to reproduce these results. You can visit this page to learn more about how CockroachDB provides predictable scaling, throughput, and latency. If you fail to achieve similar performance profiles, there is likely a problem in either the hardware, workload, or test design.
TPC-C is one of several benchmarks that we continuously run. Looking forward, these benchmarks help us identify areas for improving CockroachDB in order to provide great performance all while delivering resilience and data consistency.
